A seemingly absurd hacker experiment once again sounds the alarm for AI security. In a competition called Freysa, an AI chatbot security challenge, a hacker using the alias 'p0pular.eth' successfully 'scammed' $47,000 worth of cryptocurrency simply through well-crafted text prompts. This shocking case reveals serious security vulnerabilities present in current AI systems. The hacker's attack process can be described as textbook social engineering: first, the hacker impersonated an administrator, cleverly bypassing...

Recently, as reported by The New York Times, the internal forum of the renowned artificial intelligence company OpenAI was hacked, triggering a series of security concerns. Hackers successfully obtained detailed information about the company's artificial intelligence technology design, although they did not intrude into the systems that OpenAI actually builds AI in.This incident did not lead to any leak of customer or partner information, but it has raised concerns among employees about the comp

The JFrog security team has discovered at least 100 malicious AI ML models on the Hugging Face platform. Some of these malicious models are capable of executing code on victim machines, providing persistent backdoors. Security researchers found models with malicious functionalities among the PyTorch and TensorFlow Keras models on the platform. For instance, a model uploaded by a user named baller423 can establish a reverse shell to the host 210.117.212.93. Some malicious models may be motivated by security concerns.

Cybercriminals are starting to use generative artificial intelligence for attacks, including Meta's Llama2 large language model. The group known as Spider is using Llama2 to generate attack scripts targeting a North American financial services company. Security firms warn that the current ability to detect AI attacks is limited, but trends of malicious use of AI are expected to increase. The report notes that attempts to maliciously use large language models may still exist in 2023, but the generated results could be ineffective.

Patronus AI has released the SimpleSafetyTests suite, revealing critical security vulnerabilities in AI systems such as ChatGPT. The tests cover five high-priority harm areas, including suicide, child abuse, and bodily harm, exposing severe weaknesses in 11 LLMs. It emphasizes that secure system prompts can reduce unsafe responses, but results indicate that production systems may require additional safeguards. The testing results suggest that LLMs need rigorous and customized safety solutions before handling real-world applications.